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ABSTRACT 



This introduction to the major concepts of 
cost-benefit analysis (CBA) is intended/ for individuals involved in 
health care provider education who are attempting to measure the 
costs and benefits of their educational activities. After tracing the 
development of CBA, the paper defines it as ah attempKto quantify 
and compare the expenditures for a project with the value^of the 
finished project and identifies the ways information gained from CBA 
can be used. In addition, the paper looks at problems associated with 
the use of CBA, such as the difficulty in assigning dollar values to 
attitude changes, aesthetic values, and increased quality of life; 
the costs of the analysis; and th^ inability of CBA to take social 
^ and environmental effects into consideration. Next, a summary is 
provided of methods used to determine fixed and variable costs, and 
examples are cited of less expensive ways of providing health care 
training, such as cooperative programs, student self-training, the 
elimination of unnecessary units, and' student-financed training* The 
following section of the paper looks at ways of determining direct,/ 
indirect, and long-term benefits, considering problems in evaluating 
revenue-producing services; citing possible' assessment areas such as 
employee turnover" and effectiveness p£ training; and reviewing a CBA 
of a physical fitness program as an example of setting objectives and 
determining benefits* A glossary of terms and bibliography conclude 
the paper* (AYC)< 
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INTRODUCTION 



N Part of the economic situation of the 1980 f sis the increasing 
concern for strict budgetary accountability. As each department 
of an institution develops its goals and determines the costs of 
meeting these goals, administrators are asking for justification 
for the expenses. Cost-benefit analysis (CBA) has been of fered ,as 
a possible respcjnse to this request. This article is> intended to 0 
aid individuals involved in the education of health care provider's 
who are attempting to measure the>costs and benefit^ of their edu- 
cational activities^by introducing the ma jor • concepts in CBA. For 
specifics on "how to do CBA" readers are referred to the# -articles 
listed in the bibliography. This article is divided into five \ 

sections: / 

. I 

1. An introduction which includes the /Background and some of 
the problems with cost/berief it anal#^is\ 

2. / A summary of methods used to determine costs. 

3. A summary of methods used to determine benefits. 

4. A glossary of terms relating to economics and cost/benefit 
analysis.* ^ ? 

a t *. t 

5. A bibliography, ■ .} * 

/ Cost-benefit analysis is a Speciality of ecomonics. While the 
ilse of CBA can be traced as far- back as 1844, its popularity did 
not rise until the U.S. water programs of the 19 30's that examined 
the costs of irrigation projects and the projected increase in 
farm output that would result from the t projects (14). Today CBA is 
widely* used in investment decision-making in governmental 
(particularly the Corps of Engineers and the Bureau of 
Reclaimation) and public utility services such as transportation, 
communication, energy production, education and health. CBA has 
been so appealing that it is also being applied to small invest- 
ment decisions and in fact it? seems fashionable to do so (32). 
Health manpower training ha v s 'joined, in this trend as indicated by 
a number of recent articles on the application of CBA to health 
institution inservice education ( see references). 

In its most common usage, CBA is an attempt to quantify and . 
compare the expenditures for- a project and the value of the fin- 
ished project to allow for systematic thinking in decision •'. 
making. Traditionally, both the expenditures and the benefits are 
not only the monetary investments by and profits to the individual 
or enterprise producing the project, but also the societal cost 
and benefits (32). This societal consideration is why CBA has 
been used to justify large company projects to shareholders or 
governmental projects to taxpayers where concerns about ne$ative 
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environmental or societal impacts are raised. In many cases, use 
of CBA is mandated by government regulation as the nation attempts 
to "require managers to be more, accountable with public funds and 
provide data on the efficiency and effectiveness of service 
programs." (39) : 

Is CBA useful in institutional decision making concerning 
training in the health industry? Certainly the money invested is 
significant to warrant systematic thinking supported hf CBA. In 
1976 U.S. hospitals spent $131 million on the orientation of 
nurses and $91 million on their in-service' education . This' equals 
approximately $1*00 per patient day (21). Thus training managers 
are controlling a significant portion of the hospital industries 1 
resources and are responsible in part for the financial well-being 
of the enterprise. 

The information gained from a CBA of a training program can be 
used in support of maintaining current needs and future expansion. 
In this period of economic hardship and retrenchment, training is 
often the first to be cut back. "As profits of a company go, so 
goes training," (12) and thus the inherent good of training or its 
ability to satisfy employees is no longer sufficient to obtain ^ 
.support. Rather, training must be shown to contribute toVorgand- 
* zation objectives. CBA puts 'the evaluation of a prdgram in " ,. 
management terms and helps make it more acceptable to today's 
decision-makers (26). * 

CBA also provides assistance to the traitiing manager. It 
forces close inspection of training programs to determine if in- 
stitutional goals (benefits) aire being, met and how an educational 
experience may be contributing to meeting these goals. Discussed 
in the next section is how cost-benefit analysis allows for com- 
parison of different instructional modes to determine which mode 
is the most beneficial at the least cost. 

Terminology used in CBA is derived mostly from economics. : A 
glossary is provided in this document to help acquaint those 
starting into the literature of CBA with the terminology. One 
seeming inconsistency is the use of the phrase "bost-benef it\ 
analysis". This phrase is the most commonly used title and com- 
monly implies the ability to quantify costs and benefits in 
monetary terms and put them ihto a ratio. However, the ability to 
put many benefits into dollar terms is difficult if not impossible 
(8, 29, 32, 38, 47). Such elements as attitude changes, aesthetic 
improvements and increased quality of life are too* elusive to as- 
sign dollar values but still should be included in decision 
making. This problem is found in the analysis of benefits from 
training health professionals (and Will be discussed later) and 
has led to the. use of the -t^rm "cost^ef fectiveness" Which infers 
an analysis where the benefits are not all concrete. ■ Another 
term, "cost- justification" , is sometimes used when none, of the 
benefits are quantifiable. - 
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Cullen- et al > (13) suggests that the term "cost" is negative 
because it infers a loss to company management. They substitute 
the term investment (time and money that will eventually pay off 
in helping the company). Also, they suggest, use of the term 
"return" in place of benefit or effectiveness to encompass all 
possible gains frdm the investment. 

Drawbacks to CBA must also be considered. First is cost of 
the analysis "itself-. Time must be spent to collect and compile 
data. A $20 million dam project may justify $20,000 spent on a 
CBA, but with small investments,' the benefits in tearms of cost 
savings from - the program may be less than the cost of a complete 
analysis itself. However, model programs which are analyzed once 
and repeated additional times due to proven benefit or disconti- 
nued due to high cost compared to benefits, may warrant the in- 
spection by a CBA Also, certain assumptions can be made about 
the activity which can simplify tjhe CBA. Examples of these will 
be given later. 

*■ » ' 

.Simplification of * CBA alleviates some of the difficulty in us*- 
ing the data from a complicated CBA. The literature is filled 
with models involving mathematical formulas and statistics, yet • 
only those persons who developed the models or use them daily can 
fully understand the significance of the results. Simplified 
Methods of analysis are. not only cheaper but may be easier for 
management to ipteipret. 

Even in large scale social institutions, CBA is not always 
reliable. Jameson (20) found a dramatic/ failure of CBA in a large 
governmental department. He ascriBed the failure to 1) inappro- 
priate use of technique and - misleading guidelines for its applica- 
tion, 2) dominance of organization interest ovejr analytical 
requirements, 3) inappropriate staffing and suptoort for analysis 
and 4) use of analysis as a tool in internal struggles. This sug- 
gests a full scale CBA should be done only by qualified persons 
with sufficient support and without pressures. andv negative in- 
fluence from management. * * 

The most condemning, criticism of CBA has come in the/area for 
which it was originally designed the consideration of social ." 
and environmental effects when undertakikg aUar^e project. 
Cousins (8) states that CBA "descends like* a death sentence on any 
proposal that would apply creative imagination to socially essen- 
tial programs or long-range goals . . . ;political decisions are 
governed not by human needs or by the dictates of conscience but 
by computer printout." Kelman (23) writes that "in areas of en- 
vironmental , safety and health regulation, there may be many . in- 
stances where a certain decision might be right even though its 
benefits \don 't outweigh its cost." Augus (2) so -abhors the strict 
application of CBA that he created scenarios , based on the determi- 
nation of the cost of a human life. 

Other criticisms include the lack of consideration of scien- 
tific serendipity that may be the gart of a project but is totaljy 
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unpredictable (8); Also, CBA does not look at all situations 
equally — e.g. $600,000 to' award scholarships may provide in- 
creased^ earnings to graduates but what benefits would be obtained 
if the $600,000 was us6d to feed 50,000 Asian children? 

These complaints do' not necessarily rule out the use of CBA 
totally but rather suggest a tempering of its use. Facets of a 
project that are difficult or impossible to quantify should be 
considered seriously. Also, the type of thinking that goes into 
CBA is useful and has resulted in its use at least implicitly in 
most investment decisions. Passmore states "Attempts to specify, 
measure, and compare costs and benefits explicitly makes the deci 
sion making process available for review and critique" (29). 
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Costs are usually divided into two groups — fixed and 
variable. Fixed costs are those that will be constant regardless 
of the number of training programs or numbers of students in each 
program. Examples ' gf fixed costs are salaries of full-time staff 
and facility overhead. Variable costs are costs that may change 
based on the size aind nature of each project. Examples are con- 
sultant fees and instructional materials. It is useful to divide 
costs into these two^ groups for analysis of possible cost savings 
in a certain program. The ratio of variable to fixed costs can 'be 
revealing (47). As ,a guideline, a 2 to 1 relationship is ideal. 
If the ratio is much lower due to high fixed costs, an over- 
staffed, overequipped training department may be indicated. If 
variable costs are too low, too few participants are in the 
department offerings. If the ratio is high because fixed costs 
are low, the possibility of an ill-equipped or understaffed 
department exists. 

The challenge is to collect information on the costs of all 
factors that enter into a particular event. The following is a 
list combined from several sources (7, 26) that will help in as- 
suring a complete analysis: 

A. Fixed Cost * 

0 

1. Salaries, fringes and taxes of full-time staff (consider 
planning as well as implementation stage) " , 

2. Rent, utilities and janitorial service for staff office space 

3. Clerical assistance and supplies 

4. Equipment d . 

B. Variable * 

1% Training aids - all types of software (films, texts, 
•handouts) and hardware rented for program 

2. Reirit,' utilities, maintenance and security on space 
used for. program 

3. Consultant fees and travel 

4. Guest faculty fees, travel, room and board 

5. Printing and mailing costs of marketing materials and 
certificates 

6. * Refreshments 

7. Release time for students 

8. v Evaluation time and materials 

9. Unforeseen (margin) > ' , - , h 

• Several, of the cost factors in the list^may be difficult to 
'compute. Overhead (both fixed and variable) may not be obvious if 
it is contained within a large s institution . Estimates may. be made 
by obtaining the cotal overhead for the institution and taking a 
percentage of that figure that is based on the percent of. institu- 
tional square f dotage taken up by the educational department. 
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Staff office area would be solely based on square footage but 
classrooms used at times by other departments .would be based hot 
only on square feet but also on the length ^:time the space was 
used for training. The percentage of institutional overhead taken 
'/ f or training may also be based on the salary expenditure for the 

j department (direct labor costs) compared to the .entire institu- 

; tlonal salary costs. 

If release time,, for students needs 'to be considered, supervis- 
ors should be asked* if extra personnel, hours, overtime or compen- 
sation time are needed so cost caij be calculated. However, it is 
possible that no extra expense is incurred by time away and thus 
./ no cost would result. / 

Often the benefit of a training program is not in question 4 ' 
the program must exijst to teach needed skills or fulfill accrecft- 
tation requirements J' Thus analysis of Cost in order to compare to 
benefit is not necessary. But' cost analysis is still useful to 
determine what spending can be^ controlled or what other teaching 
methods may be less expensive , to accomplish the same goal (3/8). 
Comparisons of predicted investments for different programs; has 
resulted in the adoption of a new approach and a cost savings. 
The following are some examples of less expensive ways to /do 
training that were adopted due to cost analysis; 

1. Cooperative effort (10): Many institutions may be doing 
/ the satne program with a few attendees. * If the programs 

are" rotated from place to place, and students come from all 
the participating institutioxis, fewer programs with more 
students per program will result and drastically reduce 
co§t. A disadvantage of this approach would be travel, 
time between institutions and the coordination time in- 
volved in planning and implementation. / 

2. Training taken over by another institution: Colleges, 
universities-, proprietary groups, etc. that do training 
have the staff, facilities and equipment already on hand 
for certain activities. With training activities that are 
not needed on a regular basis or noi: utilized by large 
numbers of students, educational in-stitutions may be abie 
to do the job cheaper. This is certainly true of lengthy 
educational programs for training/ new prof essionals but 
may also be true for short programs arid continuing 
education. Costs at public. colleges and universities are 
reduced due to the tax subsidies they receive. 
Disadvantages here are tuition expense, travel time,, and 

- the necessity to conform to the schedule of the institu- 

tion offering the program. Also? control of course con- 
tent is given over to another institution. • h 

V 

3. .Students train themselves -( 22, 37, 42). Hardware and 
software for self-instruction have become commercially 
availably on a wide scale. These can be used when the 

• student has free time, thus removing release time dost.-. 
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Disadvantages are the initial cost of the equipment and 
material s, their upkeep, necessary replacement of Outdated 
material and inability to customize the instructional 
material. The last problem can be overcome by creating 
your own software; however, this is time consuming. 

Eliminate unnecessary units of training (5, 41): This as- 
sumes that a person may already be familiar with some of 
the material presented in a course. Using pretests, 
knowledge of each objective in the' instructional prograpn 
can be ascertained and students instructed only in those 
areas' in which they are deficient. This type of individu- 
ally packaged instruction is often combined with .self- 
in^tructional materials to allow the greatest flexibility 
both in time and materials. 

The major disadvantage of this "performance based staff \ 
development" is that the programs must have clear and com- 
plete objectives and the evaluation pre-tests must have 
criterion referenced> questions. The development of these 
materials can be time consuming. 

Students pay for the training (29, 41): Besides students 
directly paying tuition charges, indirect ways of shifting 
cost to the students are possible. Students' may be paid 
less on the job until they complete training ) programs , f 
concerning needed , job skills. Another mech^ism is to 
require the completion of certain types of /educational ex- 
periences for employment. \ / f 

Programs for which the attendees pay lend themselves to a 
simple CBA based solely on benefits being the revenue from 
the tuition. Once fixed arid variable costs are deter- 
mined., a graph as, shown in figure 1 may be constructed. 
The most .significant point on the graph is the point where 
the total cost line crosses the revenue, line . This indi- 
cates the number of paying participants needed to break 
even . for the program. Below that point, cost outweigh 
benefits; above that point a profit is realized. 

The disadvantage of student payment is that economically ♦ 
disadvantaged persons may Ibe held back from advancement. 
.Also, employers-support of tuition is a benefit that has a 
high positive psychologicapL impact on employees since it 
indicates desire on the pairt of the institution to improve 
its staff. 

' * ' * • ■ * . 

Make sure training is appropriate: Sovie (40) describes 
a situation wherfe errors in medication delivered to 
patients resulted in the development of a medications 
course for nurses. However* upon closer examination the 
problem was found to be getting the correct order in the 
-chart and administering the medication system. The pro- 
blem was solved when the hospital switched from team 
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- fixed cost 
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— total cost 
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Figure 1. A graph for determination of profit of 

loss on programs in which participants are 
j charged a fee, ' The graph is based on each 
attendee paying $10. (5) 
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nursing tp an individualized "system* Sovie points out 
that staff development is only a solution if 

a. a skill or knowledge, is lacking 

b. other departments are doing their job 
(i.e. - no late deliveries) 

y 

c. xtolicy and procedure are adequate and implemented 



d. the problem is remedied when the positions involved 
are staffed by trained individuals 

Thus money may be wasted if education is done when the 
problem is really a management deficit such as 

a. supportive services are making mistakes 

b. equipment and supplies are deficient or defective 

c. instructions or guidelinies are unclear 

d. policies and procedure^ need revision 

e. "health provider does hot recognize fully the perform- 
ance expectations and contributions of each discipline 
for quality care"v (i.e. doctors don f t allow nurses to 
give instructions to patients). 

Another method to assure appropriateness of training is 
the structuring of programs around j problems brought to 
light by hospital au&its (17). Audits allow the education 
to be based on Vactual statistics apd, the effectiveness of 
the training cap be monitored via changes in the audits 
after the program. 



Usq of in-hou$e Experts (15): A major cost in any training 
program is the salaries of the statf. If consultants, 
speakers, etc. are brought. in , these expenses can become 
prohibitive. Use of -persons currently on staff who have 
the knowledge 2nd ability to instruct cannot only lower 
fcosts, but also yield a better response, since the atten- 
dees know the presenter and feel more relakedan^ in- 
terested in the information. 

Np formal training at all: This may not be an option, 
but on-the-job training (OJT) has been done for years. 
Learning by experience is cost-saving and employees even- 
tually master their ^qbs* Several studies have compared 
OJT and formal training at a technical school. An air 
force study (33) concluded that technical school training 
was more costly ( $2281 \compared to $1548 for OJT) yet 60% 
of the supervisors could not detect any difference in per- 
formance and of those supervisors 1 who found a difference^ 
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there was an. even split as to wihich type tff training was 
better. Another study (12) found the cost of formal 
training to be the same as OJT (cheaper if done in large 
groups) and job skills were maste.red faster and with " 
higher competence by the formally trained group, ~Thus the 
cost savings of formalized training varies but maintains 
the advantage to being moife> consistant and controllable 
. than OJT. - - ' — ■ 

Other alternatives to reduce training costs have been sug- 
gested for industry (29) e.g. recruiting already trained individu- 
als^ relaxing requirements for quality of the workforce, or 
relocating 1 to an area where better trained workers are found. 
These options may be possible for non-service institutions, 1sut 
seem impractical for health pare. 
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BENEFITS 



Measurement of benefits in monetary terms contains many more 
complexities than measurement of costs. An article by Monat (26) 
contains a list that begins to demonstrate the variety of possible 
benefits which coulji ih^g'ained frofti training programs in an indus- 
trial situation, Ekamples of items from that list are as follows: 



1. Money- gained or saved, by increased profit due to lower labor 
use; decreased operating supplies use, reduced costs of em- 
ployee insurance , takes and benefits , fewer errors and 
. defects, decreased raw materials use, less cost reduction s 
pressure, and a decrease in attitude that people are '.less im- 
portant than sales or profits. 
* 

.2. JMon^y gained or saved by increased productivity due to faster 
order response, more efficiency, increase' production , and a 
decreased attitude that "production is obtained at any cost" 
and "employees are a means to an end". 

3. Money saved by increased quality, of work due to decreased war- 
ranty dost, decreased recall cost/ fewer complaints from 
customers, more defects caught by inspectors, lower 
defect/error rate, fewer # re jects and less of an attitude that 
"quantity is more important than quality". 

4. Money saved by better use of personnel resulting in fewer slow 
downs/stoppages*, fewer walkouts, fewer strikes, increase le£e|L 
of technical competance, lower accident frequency and severity 
rate, better development and utilization of employees, better 
decision making > a belief that effort \ is £e*?£xded, better per- 
ception of what the job is and how it is "done, fewer quits an,d" 
discharges, increased feeling that goals are achieved, less 
transfering, better suggestions, fewer grievances, less absen- 
teeism, tardiness, and higher jab satisfaction, grou# cohe- . 
siyeness, morale 1 and cooperation. 

v ~ J 

Ftom this list it is obvious that benefits can be accrued in a 
wide variety of ways. Besides the above classif i cation V another 
approach lists benefits as follows (32): 

1. Direct — chang.es ih employee performance, such as, de- 
creased waste and increased production. 

2. 'Indirect — decreased demands on supervisors and other 
employees. 

• 3. o Subsequent (long-term) - i.e. improved level of ability in 

persons presenting themselves employment. 

What is left out of fcK>th of the above are benefits to society 
$s a whole (externalities ). ^rhese a.re factors such as improved 
industrial relations, increased versatility of the workforce, dec- 
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creased crime , increased health , more jobs opening up as workers 
move up (32)', desirable behaviors, increase in non-job related 
consultation (hobbies, etc.) equity in "distribution of employment, 

higher -values and a feeling of well-being (29). 

■••/ f- 

The £ociets^>bene fits -are the hardest to quantify. Can a 
money value be placed on a feeliag^of Well-being? Attempts to do 
so have been made by surrogate pricing (32). "In this method per- 
sons a,re asked what dollar amount they would pay to have a certain 
intangible in their* lives improved i.e.,- a better view or cleaner 
air. Average values are then used in the CBA. Another approach is 
shadow pricing in which values are assigned based on best judge- 
ment .or estimate not related to market values. , . ' * 

More commonly, intangibles are not given dollar values but 
simply listed for consideration (32). Thus if a program cost 
$50, 000 and measurable benefits are" $45,000 plus intangibles ,' the / 
manager must make xf decision based on a "grey" area. A decision / 
that benefits outweigh cost could be made even if the dollar / 
amounts of taaglble benefits were less than cost. Since CBA is a 
decision r^ak^rng "tool, its usefulness is still present since all 
factors wer^ considered. 

* When/doing a CBA of .a health, institution training program, 
rarely p.re all the possible benefits measured as they are in a 
large government project. To do so would require time and money 
that could possibly negate any savings produced by the training. 
In fact many of the comprehensive analyses described in the liter- 
ature were done with the support of grant mtfney and serve as model 
projects. To be more practical, many cost-benefit analyses of 
training programs are simplified and measure one or several impor- 
tant benefits that could be used to justify the program. 

The easiest 1 type of benefit to evaluate is an increase in 
revenue, producing services (9, 30) e.g. x-rays, blood tests, or 
respiratory therapy procedures. The charge for each procedure is 
easily obtained and increase in number of procedures after train- 
ing can be acquired from departmental logs. 

Problems encountered in ;the evaluation of revenue producing ^ 
services are as follows: , 

1. Separating human contributions from technical or other 

factors^ (32v . For exantfple, 'if persons were trained to use 
a new piece of equipment and their output increased after 
training, was it the training that was responsible or the 
equipment or both? A control group that had no formal 
training with the equipment but learned on their own (from 
manuals etc.) would 7be *needed to make a clearer 
distinction. / 

] s^2. Separating the actions of one group of workers from 

another group "in a continuous process (32). For example, 
the number of procedure? done by an employee may increase 
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not because of training he received, butbecause samples 
are delivered,, faster or in better condition by another 
employee . 1 

3. Eliminating other factors in pre- and post-behavior - moni- 
toring (32). Seasonal changes, management changes etc. 
may alter output independent of training. 

4. Measuring' the life of the training investment (32).' The 
increase in output accrues over some length of time. flow 
long a time depends on other changes in equipment and per- 
sonnel that occurs. 

i ■ ' 

Decreased employee turnover has also been measured as a bene- 
fit (9,' 19, 47)". One estimate is that 300 to 700 times the hourly 
rate is spent, to replace a worker (11,). This might include lost 
production, recruiting costs and orientation. One study (36) con- 
sidered how bicultural training seminars for new nurse graduates" 
could save the hospital money due to increase retention and 
reduced orientation costs required for new employees. The results 
indicated that the seminar d'id reduce turnover and therefore more 
money was saved because of the need for fewer orientations than 
was spent conducting .the seminar.. The problem in analyzing eitt^ 



* — <- ^»v. sjv-.uu.iidj.,. -lug pj.uuj.cin in analyzing em- 
ployee turnover is in the ruling out of other factors not .related 
to training such as management changes or decreased mobility of 
the work force due to economic conditions. The use of a control 
group which does not receive the training would be necessary to' 7 
rule out these questions. • v---"""' 

In a service industry like health care, benefits are often * 
difficult to quantify because they are not immediately obvious. 
Health care may extend a patient's life as a benefit. Therefore 
•if a training program for 2 0 nurses causes each to give better 
care ' for 5 years, the result is 100 -years of better care. If 50 
patients are cared for per year by each nurse then 5000 patients 
receive better care and if the better care increases life expec- 
tancy by 10 years the final result is 50,000 years of improved 
quality of human,. life (43). Impressive figures, but the numbers 
are difficult to substantiate* 

.A more common approach is to determine, the effectiveness of a" 
program by evaluating the knowledge, skills and -attitude changes 
acquired by the participants and assuming the final result, will be 
»'an improved quality of life for the patient. This type of 

.analysis is not monetarily quantifiable and make's a traditional 
CBA impossible, but still provides information for Intelligent 
management decisions. 

t , Del Bueno (7) has ranked effectiveness- of training into five 
-categories. The lowest level of evidence is attendance only. The 
second level is an increase in awareness and/or acquired 
knowledge, skills and values. § The next level of effectiveness is 
reached if the participants, apply their acquired knowledge in a 
simulated setting. The fourth level is based on evidence that the 
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knowledge Is. applied in the real world with external reward and 
-reinforcement. The highest level assumes application and mainten- 
ance of knowledge in the real world without external 
reinforcement, Del ,Bueno then uses these levels in comparison to 
measured costs to come up with ratios" of . cost effectiveness, « 

■ ' v 'n 

How is acquisition and application of knowledge determined? 
The most common approach to the evaluation of acquisition and ap- 
plication of knowledge involves pre- and post-testing of trainees 
(4). Knowledge acquisition testing is the most common. True- 
false , fill-in ,\ multiple choice , etc. questions written specifi- 
cally to the objectives (criterion referenced) are used on pre- 
and post-test. The improvement in the Scores is the index of ef- 
fectiveness (34). Some studies follow up the post-test with a 
second test months latter to examine retention (14). » 

Increased skills ability can also be done in a pre-test/post- 
test manner in a simulated setting using mannekins, mock equipment 
set ups, etc. The use of a computer-monitdred patient simulator 
in one study enabled, the instructors to create trouble conditions 
and observe and correct cpmmon procedure errors (28). The same 
study indicated that students who did well on paper and pencil 
-knowledge tests did poorly on the patient simulator, 

Application of skills in real life has been tested via obser- 
vation and reporting. Naturalistic observation Sescribed x by 
Schinke and Wong (35) requires 2 observers to watch the interac- 
tions of a staff member with a client and rate 29 different types 
of behavior as positive , negative or neutral in response to client 
behavior which is positive, negative or neutral. This observation 
is performed for 4 tfburs in 2 pre- and post-test visits done at - 
the same time of day. This type of testing is expensive, . requires 
specially trained persons as observers and creates ..a"' certain 
amount of anxiety. Therefore it may be impractical for routine 
use. 

' J • ' 

Reporting, another method of evaluation, is o£ two types: 

supervisor and self. This method involves questionnaires 

requiring responses from the supervisor (who serves as a 

"naturalistic observer") or the employee about his job 

performance. Again these can be given as pre- and post-r 

evaluations and the difference between the two used as jbhe indica- 



cvaiuaLiuua anu me aurerence oetween rne two usea as jcne md 
tor. of effectiveness. Supervisor reporting requires the will- 
ingness of the supervisor to participate and be objective . Self- 
reporting requires ^honest objectivity from the trainee. Both 

inistered several months later to Evaluate 



would need "to. be admi 
retention. 



Examples of evaluation questions used in self-re^rting taken 
from a study by Anderson (1) are as follows: 

. 1. Do you believe the application of Clinical Instructor 

Training Techniques hais shortened the time ryeeded to train 
employees in your area of responsibility? 
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2. Do you believe the application of Clinical Instructor 
Training Techniques has helped out the cost (sic) in your 
area of responsibility? 

3. Do you" believe the application of Clinical Instructor 
Training Techniques has improved employee morals in your 
area of responsibility 

4. Do you believe the application of Clinical ;Instructor 

* Training Techniques has reduced employee turnover in your 

area? 

Another study (39) used activity reports to attempt to analyze 
benefits. Participates in the training program kept logs of 
client-related work and non-client related work (paperwork) and 
productivity joeasured before and after the training* 

Benefits such as improved attitude or job satisfaction, have 
also been evaluated via questionnaire. - One study (34) used an 
attitude checklist in which mental health workers indicated their 
feelings before and after training towards patients who were 
relaxed , withdrawn and hostile , aggressive, intelligent or conduct 
problems. A job satisfaction rating was -obtained by using check- 
lists of terms ranging from very positive to very negative and 
describing different aspects of-, their job (34),. 

The most universally used method of evaluating the effective- 
ness of a training program is the follow-up questionnaire in which 
the participant is asked to evaluate the "usefulness of the program 
to his job position. These questionnaires are simple, quick and 
serve as evidence that a training program has value. 

The analysis of^a physical fitness program by Pyle (31) serves 
as an example of setting objectives and determining benefits to be 
evaluated by one company.' The goals where divided into three 
groups. The first were short term — 3-4 months. These included 
physiological improvements such as, pulse rate, blood pressure, 
cholesterol level , and oxygen uptake. All of these improvements 
were easily measured before and after the training. Lifestyle 
questionnaires were used to determine decreases in coronary risk 
factors of participants. Short term benefits are most significant 
to^the trainees, not the company, but give a first indication the 
program is working. 

Intermediate goals (1 year) included decreased absenteeism 
among participants and an increase in moral v e, self-confidence, and 
self-image. These goals were measured by psychological tests. 
Intermediate benefits are not only good for the participant but 
also the company. • 

The last group of benefits were long-term. These benefits are 
reaped 3-5 years a,fter the training and are the most difficult to 
measure due to other factors besides" training that might be 
involved. However, long term benefits make good arguments for 
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continuing a type of training. The anticipated long term benefits 
were a decreased use of medical insurance, decreased personnel 
turnover and increased productivity of the company. 

Another example of benefit analysis in a cost-benefit study is 
the evaluation of the usefulness of students at a clinical site., 
A ^tudy of clinical training of occupational therapists was done 
by Chung (9). He used man hours in revenue producing work as his 
benefit objective. Logs, were filled out by student^ and instruc- 
tors to determine how much time was lost by the clinical instruc- 
tor due to presence of students, and how much revenue producing 
time was contributed by students. The average result was that 
students contributed 83.1% more man-hours than the agency lost. 
The results of a questionnaire on student productivity adminis- 
tered before the project are interesting. They showed that stu- 
dents thought they increased productivity by 41%. Clinical in- 
structors estimated an increased productivity of 17% and adminis- 
trators felt students decreased productivity by 25%. This indi- 
cate? clearly the pitfalls of subjective questionnaires for bene- 
fit evaluations . Even in the time logs (used for the same study) 
which should have been objective, clinical instrudtors recorded 
2.1 times more hours for clinical instruction than the students 
recorded that they received. 

A ° 
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GLOSSARY 

The following are some of the terms used in the literature 
dealing with cost-benefit analysis . As in any technical field, 
understanding the jargon means you are half the way to understand- 
ing the concepts. The following list is an aid , to that 
understanding. 

Average product o| labor - total product divided by the number of 
workers. 

Benefits - the\ positive results of an institutions activities that 
are acquired bv either the institution or society. Usually these 
results are quantifiable in monetary terms (38). ' i 

Cost - the investment in money and time an institution must make 
to create a certlain product (goods or service). 

'"■ "I *- ' A • ; - 

Criterion - any /measure of desired outcome's, results or perform.- H 

ance (26). • \ \ . . 

Dir.ect costs - expenses that are present only because o£\a spe- 
cific activity. These are sometimes divided into labor akd 
materials costs (see indirect cost). \ 

Economics - the study of how men and society choose, with or 
without the use, of money f to employ scarce productive resources to 
produce various commodities over time and distribute them for con- 
sumption , now and in the future , among various people and grdups 
in society (29). 

Effectiveness - the- positive results of an institution's activi- 
ties that are ac/quired by either the institution or society! 
Usually these results are not quantifiable in monetary terms (38). 

Evaluation - pr/ocess or \set of activities comparing results 
against goals and established critera (26)/ 

Explicit cost - capital and labor resources necessary to deliver 
.services that are usually part of budget accounting (24) (see ipi- : 
plicit cost) All direct cash payments for factor inputs 

Externalities - extra cost or benefits to society froft the produc- 
tion or consumption of goods and services (24). 

Fixed cost - costs that do not vary \in the face of changing time, 
number of students or program development ('13) (see variable 
cost) . s 1 

j ' ''■ 

Hardware, instructional - shelf items purchased to facilitate 
training program. (i.e. film projectors, tape recorders, computer 
terminals) (13) . 

. \ 
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Human capital - the capacity of an individual to produce goods and 
services (29) or just labor. 

Implicit cost - a cost that is not part of accounting process 
(i.e. costs to society due to. a person. becoming a student and 
withdrawing from the work force) (24): (see explicit cost) also 
called opportunity cost — what must be given up. 

Index - a ratio or number indicating a relationship between varia- 
ble or sets of data. ■■. 

Indirect costs - expenses incurred that are associated with a pro- 
gram but would exist even if that specific program was dropped 
(i.e. administration, facilities, full-time faculty ) ? these are 
sometimes divided into labor and materials cost. 

Initial cost - start up expenses for equipment salaries, etc., 
first time needs (36). * - 

Marginal or margin - rate of change of a factor (i.e. unit change 
of benefits gained by the input "of a certain unit of cost) (24). 

Microeconomics - study of economics of individual decision units, 
i.e. the consumer, the household, the firm; the way these units 
interrelate to determine the relative prices of goods and factors 
of production and quantities pf these which will be bought and. 
sold. Provides understanding of the mechanism by which the 
resources possessed by society are allocated among alternative 
uses. The central concept is the market (29). 

Overhead - costs of operating the physical plant of an institution 
- the fixe,d cost. 

Produbt - activities or goods that are the output of an 
institution. In health care, * patient improvement is the most com- 
mon product . 

' - ^V, 

Proportional cost - a type of variable cost that is ^directly 
related to the size of the pita je'cfc**\( i.e. a manual for each stu- 
dent) (43). 

Recurrent cost - expenses that occur each time a program is of- 
fered, usually a variable cost (36). 

Return - financial repayment of an investment. 

Shadow pricing -'a guess or estimate of a cost not necessarily 
based on market demands. 

Software, instructional - shelf items of instructional content 
that are purchased to facilitate the training program (i.e. manu- 
ls> slides, film strips) (13). 7 

illovers, social - see externalities. 
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Step cost - an expense that may remain fixed up to a certain point 
of production, then increase and remain fixed until production 
reaches another higher point (i.e. addition of an extra classroom 
if more than 30 persons attend a program) (43). 

Sunk cost - money already spent on a project (hardware) (43)]' 

.Surrogate price- - an estimate of the value of a benefit based on 
surveying those who might reap the benefit to determine what they 
rhight pay for the benefit. , . 

Total costs - variable costs plus fixed costs. 

Variable cost - those costs that vary as the time, number of stu- 
dents or 'development of a program change (13). 
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